The invention relates to a method for selecting an information source from a plurality of information sources for display on a display of smart glasses, a corresponding device and a corresponding computer program.
Nowadays, smart glasses, sometimes also called head-mounted displays, comprising a display are known. Smart glasses are worn on the head by a user in a way similar to conventional spectacles which are used as a visual aid. The display of the smart glasses is located in this case in the field of view of the wearer. Some smart glasses comprise a display only for one eye, other smart glasses comprise two partial displays, each eye being assigned a partial display.
It is known, furthermore, that smart glasses can comprise a semitransparent display. Such a display enables the wearer to recognize both displayed information and the surroundings behind the display. This type of display is particularly well suited to the contact-analog display of information. In contact-analog display, the information to be displayed is assigned a location in the surroundings of the wearer, and the information is displayed in a fashion assigned to said location, sometimes also being designated as a positionally correct display. This can be performed by seemingly superimposing the information for the wearer over the assigned location, or by displaying the information in the spatial vicinity of the assigned location, if appropriate with a symbol such as a line, in order indicate the assigned location. The information itself can likewise be a symbol, an image, text or the like. The contact-analog display is sometimes also termed augmented reality.
Smart glasses whose display does not lie in the central field of view of the wearer have become known. The field of view and the central field of view are defined by the angular range in which people can typically perceive objects in their surroundings. The angular range is determined starting from the central sightline when looking straight ahead (the central position of the pupils with reference to the eye sockets). The central field of view lies within an angular range of 10°, 15°, 20° or 25° from the central sightline when looking straight ahead. Consequently, objects of the surroundings lie in the central field of view only when they are located within a conical space of the surroundings around the central sightline when looking straight ahead. The field of view which does not lie in the central field of view is termed peripheral field of view.
By comparison with smart glasses with a display in the central field of view, smart glasses whose display does not lie in the central field of view of the wearer are less suited for contact-analog display of information. The reason for this is that when a person is performing an activity his attention and perception of his surroundings are more readily directed to the central field of view. However, smart glasses whose displays do not lie in the central field of view of the wearer offer the possibility, in particular, of providing abstract additional information to the wearer of the spectacles, such as the time of day or the date.
An object on which the invention is based is to improve the display of information by smart glasses, in particular for smart glasses whose display does not lie in the central field of view of the wearer.
This object may be achieved by the subject matter of the independent claims. Advantageous developments are defined in the dependent claims.
In one aspect, a method for determining an information source from a plurality of information sources for display on smart glasses comprises: determining the spatial orientation of the smart glasses; determining from a plurality of criteria a criterion which is fulfilled by the determined spatial orientation of the smart glasses, the criterion respectively being related to the spatial orientation of the smart glasses; each criterion being assigned an information source; selecting the information source which is assigned to the determined criterion.
This enables the wearer of the smart glasses to be presented with different information items as a function of the particular orientation of the smart glasses. Different information sources in this case supply different information, in particular different types of information, that is to say, for example, abstract variables such as the speed of a vehicle or the video image of a camera.
It frequently happens that different orientations of a person's head are associated with different activities. For example, at a typical office workplace an orientation of the head with respect to the display of a PC will be related to the person's processing of the work tasks. If the person is addressed by a colleague, the orientation of the head will change to the direction of the colleague who, in most cases, will not be located in the direction of the display of the PC. Should said person be wearing smart glasses, the spatial orientation of the smart glasses will change in accordance with the head movement of the person. This is utilized to select relevant information sources for the respective activity and display them to the person on the smart glasses. For example, the time of day can be faded in in the event of the PC display being viewed. If the person then turns his head away from the display, should there be an incoming email on the PC or an outstanding deadline it is possible to display an alert on the smart glasses which otherwise appears on the PC display and could be overlooked. In this way, the spectacle wearer is always provided with the relevant information item through the selection of the information source.
A further field of application for the method according to the invention is when driving a vehicle. Here, as well, different orientations of the head are frequently associated with different activities. When the surroundings of the vehicle, in particular the surroundings lying in front, are being viewed the head is frequently in a virtually horizontal orientation. If the driver, by contrast, is operating a function of the vehicle such as a radio or air conditioning in the dashboard, the head of the driver is frequently inclined downward. It is then possible to select, for the different orientations of the head which are associated with the different activities of the driver, information which supports the driver's activity. For example, when the driver is viewing the surroundings and holds his head in a virtually horizontal orientation, the time of day or the distance remaining to a destination can be displayed. If the driver inclines his head downward toward operating elements of the vehicle, the video recording of a front camera can be displayed by the display of the smart glasses. In this way, the driver can at least partially attend to his task of heeding the surroundings even when his view is directed into the interior of the vehicle.
Alternatively or in addition, when the driver is looking into the interior it is possible to output on the display of the smart glasses warnings by driver assist systems which have detected the danger of a collision.
The proposed method is advantageous, in particular, when the wearer of the smart glasses uses same in accordance with the typical, accepted or intended of wearing the spectacles. The plurality of information sources and the plurality of criteria can consist of two, three or more elements. One of the criteria of the plurality of criteria can be an absorption criterion which is always fulfilled when no other one of the criteria from the plurality of criteria is fulfilled.
When determining the spatial orientation, it is not mandatory to determine the complete spatial orientation in all three dimensions of space—it can likewise suffice, for example, to determine the inclination of the smart glasses relative to a horizontal.
In one development, one of the criteria from the plurality of criteria defines that the orientation of the smart glasses is such that, assuming the smart glasses are worn as intended and that they are in a vehicle, the wearer of the spectacles views the surroundings of the vehicle. Alternatively or in addition, a criterion can define that the orientation of the smart glasses is such that, assuming the smart glasses are worn as intended and that they are in a vehicle, the wearer of the spectacles perceives other traffic users, detected by a driver assist system, associated with the risk of a collision.
In another development, a criterion from the plurality of criteria specifies a range of directions; the smart glasses being assigned a direction; the method further comprising: determining the direction assigned to the smart glasses with the aid of the determined spatial orientation of the smart glasses. A range of directions is one which can, for example, be defined by specifying the angular ranges in a fashion emanating from the head of the wearer. The ranges of direction can be defined in accordance with the geometry between windows of the vehicle and the position of the head of the wearer. A range of directions can also be prescribed by a driver assist system which has detected the risk of a collision with another road user or object and has determined for the purpose of avoiding the collision a range of directions to which the driver should turn his view to detect the risk. A range of directions can also be only a stipulation of a range for the horizontal or vertical components of the assigned direction. The range of directions can take account of the settings of the seat of the wearer of the smart glasses, since said settings alter the position of the head and therefore also the range of directions, which represents a viewing of the surroundings.
In another development, the smart glasses include a camera and the determination of the spatial orientation comprises the following: receiving a photograph of the camera; recognizing the image of a portion of the interior of the vehicle in the photograph; determining the position of the image in the photograph; determining the spatial orientation of the smart glasses with the aid of the determined position of the image in the photograph. In this development, the orientation of the smart glasses is therefore obtained by evaluation of a camera photograph. This development is advantageous precisely in known surroundings such as in an interior of a vehicle. The reason is that the interior view of the vehicle is well known down to small changes owing to objects brought into the vehicle. With the aid of image processing of photographs of the camera, said knowledge can advantageously be used to determine the orientation of the smart glasses easily and accurately. Such processing methods are known in the prior art.
The recognition of the image of a portion of the interior of the vehicle in the photograph can in this case comprise: recognizing structures in the photograph; comparing the recognized structures in the photograph with stored structures of the interior of the vehicle. A structure can be formed, in particular, by the transition to windows, or by operating elements.
In a further variant, the smart glasses can, alternatively or in addition, include position sensors which are set up to detect the spatial position of the smart glasses, the determination of the spatial orientation of the smart glasses inside a vehicle then being performed with the aid of the position sensors.
In a preferred development, the plurality of information sources comprises: a video camera which is directed in the driving direction of the vehicle, or in any other direction, for example to the side of the vehicle or upward, in order to view the sky, and an information processing means which is set up to output abstract information, in particular physical measured values, of the state of the vehicle, in particular the speed of the vehicle, the rotational speed of a drive of the vehicle, the power output of the drive of the vehicle, or the filling condition of the energy store of the vehicle. It is possible to provide a plurality of video cameras between which the driver or user can switch, for example using a control surface on the smart glasses.
In one embodiment, the display of the smart glasses provides a display only for one eye of the wearer of the smart glasses. The display of the smart glasses can in this case cover less than a third or a quarter of the field of view of the eye and not cover the central field of view.
In another aspect, a device comprises: means for determining the spatial orientation of the device; electronic processing means; the device being set up to execute one of the above methods. The electronic processing means can be a computer, dedicated circuits or a microcontroller. The means for determining the spatial orientation of the device can be position sensors, or a camera in the smart glasses, and the electronic processing means, which are set up with programming technology to determine the spatial orientation. As described above, said determination of the spatial orientation can be based on the known structure of the interior of the vehicle. The device can comprise interfaces with the information sources from the plurality of information sources. Said interfaces can be hard-wired or wireless and include suitable protocols for data exchange. Such interfaces and protocols are known in the prior art.
In another aspect, a computer program comprises instructions for executing one of the above methods. The computer program can prompt a computer to execute the method when it executes the instructions.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings.
Identical reference symbols relate to corresponding elements across the figures.